Improving the arm-hand coordination in neuroprosthetics control with prior information from muscle activity

Humans use their hands mainly for grasping and manipulating objects, performing simple and dexterous tasks. The loss of a hand may significantly affect one's working status and independence in daily life. A restoration of the grasping ability is important to improve the quality of the daily life of the patients with motion disorders. Although neuroprosthetic devices restore partially the lost functionality, the user acceptance is low, possibly due to the artificial and unnatural operation of the devices. This thesis addresses this problem in reach-to-grasp motions with the development of shared control approaches that enable a seamless and more natural operation of hand prosthesis. In the first part, we focus on the identification of the grasping intention during the reach-to-grasp motion with able-bodied individuals. We propose an Electromyographic (EMG)-based learning approach that decodes the grasping intention at an early stage of reach-to-grasp motion, i.e. before the final grasp/hand pre-shape takes place. In this approach, the utilization of Echo State Networks encloses efficiently the dynamics of the muscle activation enabling a fast identification of the grasp type in real-time. We also examine the impact of different object distance and speed on the detection time and accuracy of the classifier. Although the distance from the object has no significant effect, fast motions influence significantly the performance. In the second part, we evaluate and extend our approach on four real end-users, i.e. individuals with below the elbow amputation. For addressing the variability of the EMG signals, we separate the reach-to-grasp motion into three phases, with respect to the arm extension. A multivariate analysis of variance on the muscle activity reveals significant differences among the motion phases. Additionally, we examine the classification performance on these phases and compare the performance of different pattern recognition methods. An on-line evaluation with an upper-limb prosthesis shows that the inclusion of the reaching motion in the training of the classifier improves importantly classification accuracy. In the last part of the thesis, we explore further the concept of motion phases on the EMG signals and its potentials on addressing the variability of the signals. We model the dynamic muscle contractions of each class with Gaussian distributions over the different phases of the overall motion. We extend our previous analysis providing insights on the LDA projection and quantifying the similarity of the distributions of the classes (i.e grasp types) with the Hellinger distance. We notice larger values of the Helinger distance and, thus, smaller overlaps among the classes with the segmentation to motion phases. A Linear Discriminant Analysis classifier with phase segmentation affects positively the classification accuracy

A structured prediction approach for robot imitation learning

We propose a structured prediction approach for robot imitation learning from demonstrations. Among various tools for robot imitation learning, supervised learning has been observed to have a prominent role. Structured prediction is a form of supervised learning that enables learning models to operate on output spaces with complex structures. Through the lens of structured prediction, we show how robots can learn to imitate trajectories belonging to not only Euclidean spaces but also Riemannian manifolds. Exploiting ideas from information theory, we propose a class of loss functions based on the f-divergence to measure the information loss between the demonstrated and reproduced probabilistic trajectories. Different types of f-divergence will result in different policies, which we call imitation modes. Furthermore, our approach enables the incorporation of spatial and temporal trajectory modulation, which is necessary for robots to be adaptive to the change in working conditions. We benchmark our algorithm against state-of-the-art methods in terms of trajectory reproduction and adaptation. The quantitative evaluation shows that our approach outperforms other algorithms regarding both accuracy and efficiency. We also report real-world experimental results on learning manifold trajectories in a polishing task with a KUKA LWR robot arm, illustrating the effectiveness of our algorithmic framework

Reach-to-grasp motions: Towards a dynamic classification approach for upper-limp prosthesis

During reach-to-grasp motions,the Electromyographic (EMG) activity of the arm varies depending on motion stage. The variability of the EMG signals results in low classification accuracy during the reaching phase, delaying the activation of the prosthesis. To increase the efficiency of the pattern-recognition system, we investigate the muscle activity of four individuals with below-elbow amputation performing reach-to-grasp motions and segment the arm-motion into three phases with respect to the extension of the arm. Furthermore, we model the dynamic muscle contractions of each class with Gaussian distributions over the different phases and the overall motion. We quantify of the overlap among the classes with the Hellinger distance and notice larger values and, thus, smaller overlaps among the classes with the segmentation to motion phases. A Linear Discriminant Analysis classifier with phase segmentation affects positively the classification accuracy by 6−10 on average

A Structured Prediction Approach for Robot Imitation Learning

We propose a structured prediction approach for robot imitation learning from demonstrations. Among various tools for robot imitation learning, supervised learning has been observed to have a prominent role. Structured prediction is a form of supervised learning that enables learning models to operate on output spaces with complex structures. Through the lens of structured prediction, we show how robots can learn to imitate trajectories belonging to not only Euclidean spaces but also Riemannian manifolds. Exploiting ideas from information theory, we propose a class of loss functions based on the f-divergence to measure the information loss between the demonstrated and reproduced probabilistic trajectories. Different types of f-divergence will result in different policies, which we call imitation modes. Furthermore, our approach enables the incorporation of spatial and temporal trajectory modulation, which is necessary for robots to be adaptive to the change in working conditions. We benchmark our algorithm against state-of-the-art methods in terms of trajectory reproduction and adaptation. The quantitative evaluation shows that our approach outperforms other algorithms regarding both accuracy and efficiency. We also report real-world experimental results on learning manifold trajectories in a polishing task with a KUKA LWR robot arm, illustrating the effectiveness of our algorithmic framework

EMG-Based Analysis of the Upper Limb Motion

In a human robot interaction scenario, predicting the human motion intention is essential for avoiding inconvenient delays and for a smooth reactivity of the robotic system. In particular, when dealing with hand prosthetic devices, an early estimation of the final hand gesture is crucial for a smooth control of the robotic hand. In this work we develop an electromyographic (EMG) based learning approach that decodes the grasping intention at an early stage of the reaching to grasping motion, i.e before the final grasp/hand preshape takes place. EMG electrodes are used for recording the arm muscles activities and a cyberglove is used to measure the finger joints during the reach and grasp motion. Results show that we can correctly classify with 90% accuracy for three typical grasps before the onset of the hand pre-shape. Such an early detection of the grasp intention allows to control a robotic hand simultaneously to the motion of subject's arm, hence generating no delay between the natural arm motion and the artificial hand motion

Decoding the Grasping Intention from Electromyography during Reaching Motions

Background: Active upper-limb prostheses are used to restore important hand functionalities, such as grasping. In conventional approaches, a pattern recognition system is trained over a number of static grasping gestures. However, training a classifier in a static position results in lower classification accuracy when performing dynamic motions, such as reach-to-grasp. We propose an electromyography-based learning approach that decodes the grasping intention during the reaching motion, leading to a faster and more natural response of the prosthesis. Methods and Results: Eight able-bodied subjects and four individuals with transradial amputation gave informed consent and participated in our study. All the subjects performed reach-to-grasp motions for five grasp types, while the elecromyographic (EMG) activity and the extension of the arm were recorded. We separated the reach-to-grasp motion into three phases, with respect to the extension of the arm. A multivariate analysis of variance (MANOVA) on the muscular activity revealed significant differences among the motion phases. Additionally, we examined the classification performance on these phases. We compared the performance of three different pattern recognition methods; Linear Discriminant Analysis (LDA), Support Vector Machines (SVM) with linear and non-linear kernels, and an Echo State Network (ESN) approach. Our off-line analysis shows that it is possible to have high classification performance above 80% before the end of the motion when with three-grasp types. An on-line evaluation with an upper-limb prosthesis shows that the inclusion of the reaching motion in the training of the classifier importantly improves classification accuracy and enables the detection of grasp intention early in the reaching motion. Conclusions: This method offers a more natural and intuitive control of prosthetic devices, as it will enable controlling grasp closure in synergy with the reaching motion. This work contributes to the decrease of delays between the user’s intention and the device response and improves the coordination of the device with the motion of the arm

EMG-based learning approach for estimating wrist motion

This paper proposes an EMG based learning approach for estimating the displacement along the 2-axes (abduction/adduction and flexion/extension) of the human wrist in real-time. The algorithm extracts features from the EMG electrodes on the upper and forearm and uses Support Vector Regression to estimate the intended displacement of the wrist. Using data recorded with the arm outstretched in various locations in space, we train the algorithm so as to allow robust prediction even when the subject moves his/her arm across several positions in space. The proposed approach was tested on five healthy subjects and showed that a R2 index of 63:6% is obtained for generalization across different arm positions and wrist joint angles

Customizing skills for assistive robotic manipulators, an inverse reinforcement learning approach with error-related potentials

Robotic assistance via motorized robotic arm manipulators can be of valuable assistance to individuals with upper-limb motor disabilities. Brain-computer interfaces (BCI) offer an intuitive means to control such assistive robotic manipulators. However, BCI performance may vary due to the non-stationary nature of the electroencephalogram (EEG) signals. It, hence, cannot be used safely for controlling tasks where errors may be detrimental to the user. Avoiding obstacles is one such task. As there exist many techniques to avoid obstacles in robotics, we propose to give the control to the robot to avoid obstacles and to leave to the user the choice of the robot behavior to do so a matter of personal preference as some users may be more daring while others more careful. We enable the users to train the robot controller to adapt its way to approach obstacles relying on BCI that detects error-related potentials (ErrP), indicative of the user’s error expectation of the robot’s current strategy to meet their preferences. Gaussian process-based inverse reinforcement learning, in combination with the ErrP-BCI, infers the user’s preference and updates the obstacle avoidance controller so as to generate personalized robot trajectories. We validate the approach in experiments with thirteen able-bodied subjects using a robotic arm that picks up, places and avoids real-life objects. Results show that the algorithm can learn user’s preference and adapt the robot behavior rapidly using less than five demonstrations not necessarily optimal

Decoding the grasping intention from electromyography during reaching motions

Abstract Background Active upper-limb prostheses are used to restore important hand functionalities, such as grasping. In conventional approaches, a pattern recognition system is trained over a number of static grasping gestures. However, training a classifier in a static position results in lower classification accuracy when performing dynamic motions, such as reach-to-grasp. We propose an electromyography-based learning approach that decodes the grasping intention during the reaching motion, leading to a faster and more natural response of the prosthesis. Methods and Results Eight able-bodied subjects and four individuals with transradial amputation gave informed consent and participated in our study. All the subjects performed reach-to-grasp motions for five grasp types, while the elecromyographic (EMG) activity and the extension of the arm were recorded. We separated the reach-to-grasp motion into three phases, with respect to the extension of the arm. A multivariate analysis of variance (MANOVA) on the muscular activity revealed significant differences among the motion phases. Additionally, we examined the classification performance on these phases. We compared the performance of three different pattern recognition methods; Linear Discriminant Analysis (LDA), Support Vector Machines (SVM) with linear and non-linear kernels, and an Echo State Network (ESN) approach. Our off-line analysis shows that it is possible to have high classification performance above 80% before the end of the motion when with three-grasp types. An on-line evaluation with an upper-limb prosthesis shows that the inclusion of the reaching motion in the training of the classifier importantly improves classification accuracy and enables the detection of grasp intention early in the reaching motion. Conclusions This method offers a more natural and intuitive control of prosthetic devices, as it will enable controlling grasp closure in synergy with the reaching motion. This work contributes to the decrease of delays between the user’s intention and the device response and improves the coordination of the device with the motion of the arm